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Abstract 

Measurement and evaluation of students’ achievement are an important aspect to make sure that student really 
understand the course content and monitor students' achievement level. Performance is not only reflected from 
the numbers of high achievers of the students, but also on quality of the grade obtained; does the grade 'A' truly 
reflective of a high achiever student. As part of quality improvement, standard setting for students’ examination 
scoring should be set up for courses offered in the institutions. Setting the cut scores of the performance standard 
is important in making sure the performance is of expected standard. Commonly standard settings done through 
experts' judgement based on their knowledge and experience on the subject's matter and estimation made often 
too difficult and confusing for many. This paper discusses implementation of Objective Standard Setting (OSS) 
using Rasch measurement model in a Malaysian Institution of Higher Learning. This method gives an advantage 
for academic administrators to scientifically establish the standards or cut scores focusing on validity of the test 
being used and the Rasch measurement properties of the resultant scale. 

Keywords: objective standard settings, measurement and evaluation, Rasch measurement model, systematic 

1. Introduction 

1.1 Introduce 

Engineering programs in Malaysia have been progressively going through changes in making sure higher 
learning institutions produced quality graduates fit for the working environment. Graduates not only obtained 
good grades but are being prepared for the industry through trainings as well. It is in line with the aims of the 
ministry who wants to produce graduates who are competent and able to obtained employment in the relevant 
field. In fact, the Ministry of Higher Education (MOHE) has formulated a strategic plan for nurturing a 
knowledgeable, innovative and first class human capital in 2020. The plan emphasizes on the improving quality 
of teaching and learning including enhancing the system of student assessment by paying attention to the 
students’ achievement other than the writing test (http://www.mohe.gov.my/psptn/). 

Besides MOHE, the Malaysian Qualifications Agency (MQA) provided the standards and framework for 
engineering programs in making sure that the institutions of higher learning are able to produce quality graduates 
(MQA, 2011). MQA also ensures quality of the courses offered either in local (IPTA) or private institutions of 
higher learning (IPTS). Code of Practice for Programme Accreditation (COPPA) and Code of Practice for 
Institutional Audit (COPIA) were developed to assess the courses offered in higher education institutions. Some 
key aspects which are guaranteed by MQA are a continuous quality improvement and student assessment 
(http://www.mqa.gov.my). Besides ensuring the goal and learning outcomes of the courses are met, students’ 
achievement also are being monitored to produce competitive and quality students in the face of global 
competition in line with the aspirations and the National Education Philosophy. Measurement, evaluation and 
assessment were done with continuous effort in meeting the expected outcomes, without foregoing quality. 

At the same time, Faculty of Engineering & Built Environment (FKAB) performs a variety of teaching and 
learning methods to meet the needs of the industry hence accordance with accreditation requirements. 
Subsequently, FKAB make improvements from time to time to make changes in the methods of teaching and 
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learning in stages. Measurement of quality of students always been discussed especially in determining the 
achievement lines. Not only in IPTA and IPTS, Ministry of Education also agrees with the importance of 
achievement lines. In year 2012, only 3,000 among 12,000 polytechnics graduates offered to further their studies 
in local institutions of higher learning. It is because the scoring standard, Cumulative Grade Point Average is 
lower than the Matriculation and Malaysian Higher School Certificate (STPM) colleges. Because of that issue, 
the Ministry of Education has called all polytechnics' management to revise the examination standard and 
grading system to ensure their graduates managed to further their study in local universities (Utusan, 2012). 

Given that this issue arose, it clearly shows that setting the benchmark standard for the students to pass the exam 
is also important to improve quality of teaching and learning, thus improving quality of students in an 
educational institution. Quality assessment is expected to produce credible information that facilitates critical 
decision making (Zamaliah et al., 2011; Stone, 1995). It depends on what are expected and calibrated 
instruments that provide the test-takers various levels of measures for decision makers to judge what is best and 
less achievement (Wright, 2000). It has always been an issue when deciding where the lines of achievement 
should be, taking consideration of students’ measures and quality that been set. 

2. Standard Setting Review 

The process of teaching and learning is expanding and the standard setting procedure is one of the important 
aspects to improve teaching and learning. In preserving and ensure the test results will be useful and defensible, 
standard setting ideally involves with policy makers, developers test and measurement expert, not just a 
methodological process (Bejar, 2008). 

Standard setting can be defmeas a procedure of establishing one or single cut score (such as pass/fail and 
allow/deny) or more (multiple) cut score (e.g. level of achievement such as weak, moderate, excellent) in a 
single test depends on what test requirement. This cut score function as separation of two or more categories 
required by the test (Cizek & Bunch, 2007). Meanwhile, other definitions of standard setting besides determines 
the level of achievement or mastery is a method applied to obtain the corresponding cut score which can classify 
students who are under cut score to a higher level (Bejar, 2008). 

There are many procedures for standard setting such as Angoff, Nedelsky, Ebel, Bookmark and Objective 
Standard Setting. Abu Kassim (2007) argued that educational standard is lack of objectivity due to the 
unsystematic judgement of the experts when constructing the cut scores. Angoff had its way by averaging the 
subject-matter experts predicted difficulty of the items (James et al., 1998) in which described by Sick (2009) as 
“too difficult and confusing”. Objective (OSS) model and Bookmark represent clearly departures from Angoff 
model, Ebeland Nedelsky (Stone, 2011). Compared with both model, Angoff failed to define a legitimate and 
stable construct and avoids the item by item judgement may be tedious and difficult to judges (Stone, 2011; 
MacCann & Stanley, 2006). 

Objective Standard Setting through Rasch measurement model has shown efficacy in psychometric of the stable 
formation of more reasonable standard (Stone, 1995). It can lead to the acceptable benchmark of the test taker. 
This method asks the experts who act as judges to determine the areas of knowledge required directly from the 
items or questions in the test. These experts will also need to set standard set for the course or subjects, at 
appropriate benchmark on the measuring line between essentialand higher items. 

This paper explores the objective standard setting with constructed-response items with the use of a 
research-based approach in the quantification of the judges’ ratings. 

3. Method 

Objective Standard Setting method is based on the study conducted by Wright and Grosse (Wright & Grosse, 
1993; Wright & Grosse, 1987). This method combines between expert judgement on the test content and 
consideration of the ability and the difficulty of students-item respectively. 

3.1 Objective Standard Setting Methodology 

This method involves evaluative decisions and translation in the quantification of the standard. The evaluative 
decisions steps are as below; 

a) Expect judgement on essential and optional items within the test. 

b) The level of mastery required and the quantification of the standards. 

a) Calculation of the mean item difficulty for all the essential items within respective grade separation; ja 
difficulty. 
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b) Translating the mastery level decision onto the logit measurement. 

c) Calculating the standard error (SE) of each grade separation; for the allowance of grade demarcation changes. 
The steps to fix the cut score is outlined are; 

a) Judges to determine the essentiality of items in the prescribed test; E item, thus establishing total of E items; 
2N e 

b) Considering the number of grade separation, and to divide the number of total essential items with the 
number of grade separation; ZN E /N S to give the number of items per grade. 

c) Calculate the mean difficulty of essential items within each grade; ju it em. Include also calculation for mean SE; 

Pse. 

d) Decision by the expert to set the ‘mastery’ level, which is normally set at 60%. Therefore, 60% out of the 
total number of essential items would give the number of items actually needed to achieve the mastery level 
taking consideration from the easiest essential items, in ascending order from item measure table, thus having 
the logit measurement. 

Upon having the logit measure on accepTable 30% essential items, then map it on to the person measure table; to 
consider the number of accepted students in mastery level. 

3.2 Participant 

217 of the first year students of engineering faculty at a Malaysian institution of higher learning involved in this 
study. It comprises of four (4) disciplines of the engineering program; civil structural, mechanical, chemistry 
process, electric electronic. The students distributions based on department and gender are shown in Table 1. 


Table 1. Distribution based on department and gender 




N 

% 

Department 

Civil & structural 

46 

21.2 


Electric & electronic 

56 

25.8 


Chemical & process 

47 

21.7 


Mechanics & materials 

68 

31.3 

Gender 

Male 

134 

61.8 


Female 

83 

38.2 


3.3 Instruments 

The students’ raw scores gathered from final examination. 19 items of linear algebra (KKKM1224) final 
examination paper were use in this study. Linear Algebra is one of the engineering mathematics subjects in 
UKM which discussed on complex number, hyperbolic functions, power series expansion, matrix and 
determinant and important applications of linear algebra in engineering. This course will highlight to the students 
on importance of linear algebra in engineering (Undergraduate Handbook, 2011). All items were given to judges 
to review their essentiality. Essential items refer to the major item which significant to achieve learning outcome 
and meet the objective of the course or subject. The details of test items are summarise in Table 2. 
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Table 2. Topics for each examination question 

Part Question. No Marks 


A1 

Determinant matrix 

10 

A2 

Positive integer 

10 

A3 

Eigenvectors 

12 

A4 

Diagonalizable 

8 

B5 

Transition matrix 

9 

B6 

Vector transition 

4 

B7 

Linear combination vector 

4 

B8 

Graphic linear 

3 

C9 

Subspace 2x2 

9 

CIO 

Subset of subspace 

6 

Cll 

Basis of subspace 

4 

C12 

Not subspace 

1 

C13 

Orthogonal product 

2 

C14 

Evaluate orthogonal 

4 

C15 

Gram-Schmidt orthogonal 

14 

C16 

Elastic deformation 

8 

C17 

Deformation boundary 

2 

C18 

Conic quadratic 

8 

C19 

Quadratic graph 

2 


3. Results 

The raw scores were then run using Winsteps version 3.68.2, a Rasch analysis software. To ensure that there are 
no misfit items which can affect the result for standard setting, the test items need verification procedure before 
proceed with OSS procedure. In Rasch analysis, there are three-step comparison procedures to determine the test 
items misfit. The procedure start with point measure correlation (x), followed by outfit MNSQ and then with 
outfit ZSTD (z) value. Each control have accepted region given as follows; 0.4<x<0.8, 0.5<MNSQ<1.5 and 
-2.0<z<2.0. Item misfit if all controls did not meet the accepted region respectively. Figure 1 shows the 
Item-Measure Order. 
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Item STATISTICS: MEASURE ORDER 


| ENTRY 

TOTAL 



MODEL | INFIT 

| OUTFIT | PT-MEASURE | EXACT 

MATCH| 



1 

INUMBER 

SCORE 

COUNT 

MEASURE 

S.E. IMNSQ 

ZSTDIMNSQ 

ZSTD | CORR. 

EXP. | 

OBS% 

EXP% | 

Item 

G 

1 

1 13 

91 

194 

1.27 

.081 .71 

-1.3 

.86 


-.5 

.22 

.30 | 

58.8 

61.0| 

C13 Orthogonal product 

1 

1 

1 

1 11 

14 

46 

1.08 

.1811.20 

.6 

1.04 


.3 

.31 

.38 | 

80.4 

78.6| 

Cll Basis of subspace 

0 

1 

1 8 

238 

217 

.40 

.061 .95 

-.8 

.94 


-.7 

.40 

.34| 

15.2 

17.6| 

B8 Graphic linear 

F 

1 

1 19 

152 

192 

.35 

.08| .89 

-2.1 

.87 


-2.3 

.46 

.261 

22.9 

18.8 1 

C19 Quadratic graph 

0 

1 

1 17 

195 

192 

.07 

.08| .90 

-2.3 

.89 


-2.4 

.44 

.26| 

17.7 

17.4 1 

C17 Deformation boundary 

0 

1 

1 7 

461 

217 

.02 

.041 .92 

-1.3 

.91 


-1.2 

.49 

-41| 

16.1 

16.91 

B7 Linear combination vector 

0 

1 

1 6 

468 

217 

.01 

.04| .87 

-2.3 

.83 


-1.9 

.54 

.44 | 

10.6 

11.0| 

B6 Vector transition 

7 

1 

1 3 

1603 

217 

-.03 

.0311.03 

.3 

1.02 


.3 

.60 

.60| 

14.7 

13.91 

A3 Eigenvectors 

B 

1 

1 5 

1156 

217 

-.04 

.0211.08 

1.0 

1.01 


.1 

.60 

.611 

16.1 

16.8 1 

B5 Transition matrix 

9 

1 

1 4 

1135 

217 

-.08 

.0311.15 

1.6 

1.38 


3.0 

.43 

.50 | 

22.1 

20.0 1 

A4 Diagonalizable 

1 

1 

1 9 

258 

46 

-.12 

.0511.09 

.5 

1.40 


.9 

.55 

.581 

26.1 

24.61 

C9 Subspace 2x2 

0 

1 

1 16 

1102 

192 

-.13 

.04| .99 

.0 

.94 


-.3 

.51 

.49| 

25.0 

23.5| 

C16 Elastic deformation 

0 

1 

1 18 

1179 

192 

-.19 

.03| .99 

-.1 

.90 


-.4 

.51 

.48 | 

34.4 

26.91 

Cl8 Conic quadratic 

0 

1 

1 15 

1940 

194 

-.22 

.0211.08 

.8 

1.22 


1.7 

.59 

.63 | 

8.2 

11.11 

C15 Gram-Schmidt orthogonal 

J 

1 

I 1 

1500 

217 

-.35 

.0311.31 

3.4 

1.43 


3.3 

.39 

.56| 

12.9 

15.11 

Al Determinant matrix 

2 

1 

1 10 

223 

46 

-.35 

.07| .91 

-.3 

.56 


.1 

.47 

.42 1 

37.0 

34.3 1 

CIO Subset of subspace 

K 

1 

1 12 

28 

46 

-.40 

.31| .95 

-.7 

.94 


-.6 

.31 

-17| 

67.4 

61.5| 

C12 Not subspace 

H 

1 

1 14 

661 

194 

-.55 

.061 .91 

-.7 

.76 


-1.1 

.43 

.33 | 

52.1 

49.4 1 

Cl4 Evaluate orthogonal 

0 

1 

1 2 

1980 

217 

-.74 

.0411.00 

.1 

.90 


-.4 

.41 

.40| 

35.9 

36.1| 

A2 Positive integer 

0 

1 

| 




- 1 - 






- 


- 



■ | 

| MEAN 

757.1 

172.1 

.00 

.0711.00 

-.2 





1 

30.2 

29.2 1 



1 

| S.D. 

647.1 

66.0 

.48 

.07| .13 

1.4 

2 


3 

1 

1 

20.1 

19.0 1 



1 








Figure 1. Item-Measure order 


From Item-Measure order table, there are no misfit items in this examination question paper. Then, we can 
proceed to the OSS procedure with all 19 items. 

Raw scores always mislead our perception on students’ ability. It does not indicate the marks earned are from 
difficult items to confirm their valid ability. Figure 2 shows that student 197 with score 93, student 65 with score 
91, student 67 with score 86 are better ability students as compared to their corresponding higher scores students 
namely student 29 with score 96, student 138 with score 92, student 16 with score of 87. The latter students 
obtained higher scores from easy items, where else the former, despite having low scores but obtained the marks 
from more difficult items. Rasch position them of better ability according to their ascending ability order. Raw 
scores is only indication of achievement but not a measure of ability. OSS gives a better psychometry of the 
students’ ability. 


|ENTRY 

TOTAL 

TOTAL 


MODEL| 

1 


|NUMBER 

SCORE 

COUNT 

MEASURE 

S.E. | 

1 

Person 

i 




1 

_L . 






T 

1 


1 1 197 

93 1 

16 

1.03 

.32| 

1 

1974MC 

| 29 

96 

15 

1.02 

.27 | 

1 

0291FC 

| 113 

95 

15 

.95 

.27 | 

1 

1132MM 

1 1 60 

94 

15 

.94 

.25| 

1 

1603FC 

1 1 65 

91 1 

16 

.85 

.29| 

1 

0652MC 

| 138 

92 

15 

.81 

.25| 

1 

1383MC 

| 100 

92 

15 

.75 

.25| 

1 

1002FC 

| 58 

89 

16 

.70 

.26| 

1 

0582MC 

| 190 

91 

15 

.69 

.24 | 

1 

1904MC 

1 61 

90 

15 

.63 

.23| 

1 

0612MC 

| 178 

90 

15 

.63 

.23| 

1 

1784MC 

| 105 

89 

15 

.58 

.22| 

1 

1052MM 

| 174 

88 

15 

.58 

.22 | 

1 

1744MC 

1 4 

88 

15 

.53 

.211 

1 

0041FM 

1 182 

88 

15 

.53 

.211 

1 

1824MM 

1 1 67 

86 | 

16 

.53 

.22| 

1 

0672MC 

| 16 

87 

15 

.49 

.20| 

1 

0161FC 

1 51 

87 

15 

.49 

.20| 

1 

0512MM 


Figure 2. Snippet of the person measure table 


Setting the standards starts by identifying the essential items among the items prescribed in the test. Experts in 
the area of linear algebra are to identify essential items within the 19 items in the test. This is crucial in 
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determining the topics which deemed important and for the students to achieve in mastering linear algebra. For 
the purpose of this paper, an expert has identified 16 items as essential items. Those items indicated with 4 E’ in 
Figure 3. 


I ENTRY 
I NUMBER 


TOTAL TOTAL 
SCORE COUNT 


I MODEL I 

MEASURE S.E. h 


INF IT | OUTFIT I FT -MEASURE I EXACT MATCH I 
HN3Q ZSTDIMNSQ ZSTDICORR. EXP. | OES% EXFI | Item 
+----+ - 


B 13 

91 

194 

1.27 

.□8 

.71 

-1-3 | 

86 

-.5| 

.2: 

1 11 

14 

46 

l.DB 

.18 

1.2D 

.611 

04 

.31 

. 3 

1 s 

23 S 

217 1 

.40 

.05 

. 95 

-.SI . 

94 

-.7 | 

. 41 

I 19 

15 2 

192 

.35 

.OS 

. S9 

-2.11 . 

, 37 

-2.3| 

-4| 



.OS 

. 90 

-2.31 . 

S9 

-2.4 1 








Group 7 


.04 .92 -1.31 .91 -1.21 ■ 4 GrOUD 6 16-91 B7 Linear combination vector E 

.04 .07 -2.31 .03 -1.91 .5 v 11.DI B6 Vector transition E 





BS Graphic linear 
Cl9 Quadratic graph 
Cl7 Deformation bcundar 


Transition Tr.striE 


24.61 C9 Sub space 2x2 


23.51 C15 Elastic deformation 
25.91 CIS Conic quadratic 
11.11 C15 Gram-Schmidt ortho-go 


15.11 A1 Determinant matrix E 

34.31 CIO Subset of subspace E 


51.51 C12 Not subspace E 

49,41 C14 Evaluate orthogonal E 

35.11 A2 Positive integer 


I 30.2 29.21 

I 20.1 19.01 


Figure 3. Item measure table 


Usually, the grading system has 5 separations; A, B, C, D, F. However, due to leniency of grading, this academic 
institution has 11 separations; A, A-, B+, B, B-, C+, C, C-, D+, D, F; then dividing all the items in the test with 
the grades separations yielding approximately about 2 items for each grade. The separation should starts from the 
lowest item in measure table as in Figure 3 which sorted according to their respective difficulty logit measure 
with the lowest is the easiest items. 

Distribution of essential items according to the 11 grading system causing the items being segregated into 8 
groups altogether. Table 3 list out the items according to their respective groups and difficulty measures, from 
the lowest; 


Table 3. Groups and respective logit measures 



Items 

Logit measure 

Group 1 

C14 

-0.55 


C12 

-0.40 

Group 2 

CIO 

-0.35 


A1 

-0.35 

Group 3 

C15 

-0.22 


C16 

-0.13 

Group 4 

C9 

-0.12 


A4 

-0.08 

Group 5 

B5 

-0.04 


A3 

-0.03 

Group 6 

B6 

+0.01 


B7 

+0.02 

Group 7 

C17 

+0.07 


B8 

+0.40 

Group 8 

Cll 

+1.08 


C13 

+1.27 


The logit measures for all the items within each groups are then summed up and divided by the number of items 
for each group giving the mean value for each group. The mean value will act as the cut score value for that 
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particular group. Similarly, mean of each standard error has calculated for each group. Table 4 shows the list of 
groups with mean and mean error. The mean standard error is allowance of range systematically provided in 
situations where decision need be made on reducing or increasing the cut score for each group grades. 


Table 4. Groups with mean and mean error 



Mean logit; jli 

jliS.E 

Group 1 

-0.48 

0.19 

F 



Group 2 

-0.35 

0.05 

D 



Group 3 

-0.18 

0.03 

D+ 

Group 4 

C- 

-0.10 

0.03 

Group 5 

-0.04 

0.03 

C 



Group 6 

0.02 

0.04 

C+ 



Group 7 

0.24 

0.07 

B- 



Group 8 

1.18 

0.13 

B 




However, refer to Figure 3 again, take note that some of the items have similar logit measures and some have 
narrow measures between the items. For example items in Group 2, both items; A1 Determinant matrix and 
ClOSubset of subspace, both having same measures at -0.35 logit. Rasch analysis provides empirical evidence 
that the items are redundant. Lucky enough that both items are from different topics or dimension. 

Items in Group 5 and 6, which the measures are so narrow, are vague in their grade separation. Considering the 
measurement for item B6_Vector transition at +0.01, SE 0.04; on the high side the item could be located 
at+0.05 logit and the lower end at -0.03 logit. At the worst scenario, the item has crossed over the Group 
6 separation instead. This is another consideration to note before embarking to OSS where items need calibrated 
properly. However, for this exercise in view of insufficient items, so the process has skipped. 

The students expected to meet 60% ‘mastery’ level of among the essential items in order for them 
being considered as meeting the objective of linear algebra subject. This gives the demarcation point of roughly 
meeting or achieving 10 out of the essential 16 items; marked at -0.03 logitmeasure or at A3_Eigenvectors item. 
Mapping the mean logit measures on to the person measure table where the person sorted in ascending order 
according to their respective ability measures. 
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Figure 4. Snippet of person measure table 


ENTRY ! TOTAL ! TOTAL ! 


MODEL ! 



NUMBER 1 SCORE 1 COUNT 1 MEASURE 1 

S.E 1 

NAME 1 

1 

197 I 

93 - 

16 - 

1.03 ! 

0.32 ; 

1974MC I 


201 1 

79 1 

15 1 

0.25 1 

0.15 1 

2014MM 1 

B- 1 

2 | 

1 

78 I 

1 

15 I 

1 

0.23 I 

0.15 i 

l 

0021MM i 

1 


1 

152 | 

i 

66 , 

i 

15 , 

0.02 , 

0.12 | 

1523FI | 

C+ [ 

36 1 

65 1 

15 1 

0.01 [ 

0.12 1 

0361FM 1 

C 1 

62 | 

64 | 

15 i 

0.01 i 

0.12 | 

0622MC i 


94 1 

65 1 

15 1 

0.01 1 

0.12 1 

0 942 FM 1 


142 1 

65 I 

15 I 

0.01 I 

0.12 | 

1423 FM , 


154 1 

65 l 

15 ' 

0.01 ■ 

0.12 l 

1543 FM l 


177 | 

64 [ 

15 ! 

0.01 [ 

0.12 ] 

1774MC | 


161 l 

63 I 

16 l 

0 i 

0.12 1 

1613MC 1 


168 ! 

63 ! 

16 ! 

0 [ 

0.12 | 

1684MM | 


37 l 

64 i 

15 i 

-0.01 i 

0.12 i 

0371MM i 


43 1 

64 1 

15 1 

-0.01 1 

0.12 | 

0431FM 1 


72 i 

62 i 

15 i 

-0.01 i 

0.11 i 

0722MM i 


156 1 

64 1 

15 1 

-0.01 1 

0.12 1 

15 63 FM 1 


205 | 

64 [ 

15 l 

-0.01 I 

0.12 \ 

2054MM | 


9 1 

63 l 

15 ' 

-0.02 I 

0.12 l 

0091FM l 


26 1 

63 I 

15 ! 

-0.02 [ 

0.12 [ 

0261MM , 



Cut score 60% at 

-0.03 

-0.02 i 

0.12 I 

0 972 FM i 






-0.02 [ 

0.12 ' 

1233FC ! 


i 

130 1 

i 

63 1 

i 

15 i 

-0.02 , 

0.12 i 

1303 FM i 


_170_ | 

. _ 63 L . 

15 L ! 


L 0 - 1 ! 1. 

1704 FM 1 


5 i 

62 i 

15 i 

-0.04 i 

0.12 I 

0051FM 1 


23 | 
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Figure 4 of the person measure reveals that the ‘mastery’ point is within ‘C’ group, however, five (5) of the 
students did not reach the -0.03 logit measures even though there are within ‘C’ group in the conventional 
fixed-grade setting. This reveals, even though that the five (5) students are within grade ‘C’ but they do not 
reached ‘mastery’ acceptance of linear algebra; they may only managed to answer correctly on easy items and 
have some difficulties answering difficult items. 

Having considering passing ‘mastery’ level of linear algebra at 60% which marked at -0.03 logit, revealing that 
139 out of 217 students managed to reached the mastery level. This is equal to 64% passes. However, if the 
academic administrators decide to have high number of passes, the SE value could be use. Rasch applies the SE 
of measures as the limit for variance. Referring to Table 5, the 60% mastery level measure is at -0.03 logit, SE 
0.03, will have a lower limit of -0.06 logit. This allows 12 students for possible inclusion as deemed mastery. 
Now the number of passes has increased to 151 (from 139) or 70% (from 60%). The academic administrators 
can report such achievement at high 70% of mastery skill achieved and yet does not breach the ethics and values 
of morality. This is contrary to many practices to achieve higher percentages of passes by arbitrarily increased 
the marks of all students. 


158 































www.ccsenet.org/ies 


International Education Studies 


Vol. 6, No. 6; 2013 


However, the exercise in increasing the number of passes allowed the grade C- students to pass the test. 
Therefore, here, the academic administrators have to decide whether to allow the increase of passes but on other 
hand jeopardizing quality of the ‘passes’. 

4. Conclusions 

The Ministry of Higher Education has formulated a strategic plan to improve quality of teaching and learning in 
effort producing quality and competitive students. In the meantime, education standards have always been an 
issue of learning institutions where it involved experts’ judgments that is still chaotic in some ways. Judges 
perception on essentiality items may vary depending on their scope requirement, knowledge and expectation of 
their expert nature. Objective Standard Setting (OSS) using Rasch measurement model facilitates the various 
perceptions of experts in deciding systematically on the cut-off line of mastery and performance levels of the 
test-takers without jeopardizing quality of the performance. Rasch logit can measure students' ability and an 
approach where the decision is being more objectively on measure scale (Zamaliah et al., 2011). This method 
could be used by academics and the grade coordinator as an alternative method for each course offered at 
education institutions. This method is one of the efforts to overcome factors affecting examination quality. 
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